
Ztf'vs^*^ United States Patent and Trademark Office 



UNITED STATES DEPARTMENT OF COMMERCE 
United States Patent and Trademark Office 
Address: COMMISSIONER FOR PATENTS 
P.O. Box 1450 

Alexandria, Virginia 22313-1450 
www.usplo.gov 



APPLICATION NO. 



FILING DATE 



FIRST NAMED INVENTOR 



ATTORNEY DOCKET NO. CONFIRMATION NO. 



09/703,423 



11/01/2000 



Antonio J. Colmenarez 



US 000273 



1410 



24737 7590 02/25/2004 

PHILIPS INTELLECTUAL PROPERTY & STANDARDS 
P.O. BOX 3001 

BRIARCLIFF MANOR, NY 10510 



EXAMINER 



LU, TOM Y 



ART UNIT 



PAPER NUMBER 



2621 

DATE MAILED: 02/25/2004 



i 



Please find below and/or attached an Office communication concerning this application or proceeding. 



PTO-90C (Rev. 10/03) 



• 

Office Action Summary 


Application No. 

09/703,423 


Applicant(s) 

COLMENAREZ ET AL 




Examiner 

Tom Y Lu 


Art Unit 

2621 







The MAILING DATE of this communication appears on the cover sheet with the correspondence address - 



Period for Reply 
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DETAILED ACTION 
Response to Amendment 

1 . The written response and amendment filed on December 1 5, 2003 has been entered. 

2. Claims 1-21 are pending. 

Response to Arguments 

3. Applicant's arguments with respect to claims 1, 20 and 21 have been considered but are 
moot in view of the new ground(s) of rejection. 

The Brill Reference: 

Applicant argues the Brill reference does not teach the newly amended limitation of 
"wherein the optimum trajectory of a person is maximized through linear transformation". As a 
result, the Brill reference fails to anticipate all the elements cited in the claims. 

Upon further review of specification, and in light of applicant's arguments, the examiner 
agrees the Brill reference does not explicitly teach use of linear transformation to maximize the 
optimum trajectory of a person. Nonetheless, such technique is well known in the art. 
Accordingly, a new reference is cited in the following office action. 

Claim Rejections - 35 USC § 112 

The following is a quotation of the first paragraph of 35 U.S.C. 1 12: 

The specification shall contain a written description of the invention, and of the manner and process of making 
and using it, in such full, clear, concise, and exact terms as to enable any person skilled in the art to which it 
pertains, or with which it is most nearly connected, to make and use the same and shall set forth the best mode 
contemplated by the inventor of carrying out his invention. 

4. Claims 1-14 and 18-21 are rejected under 35 U.S.C. 112, first paragraph, as failing to 
comply with the written description requirement. The claim(s) contains subject matter which 
was not described in the specification in such a way as to reasonably convey to one skilled in the 
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relevant art that the inventor(s), at the time the application was filed, had possession of the 
claimed invention. 

a. Referring to Claim 1, the examiner does not find any explanation in the 
specification with regard to the limitation of "the optimum trajectory of a person 
is maximized through linear transformation". The closest explanations given in 
the specification, in page 6, lines 10-15, are "the position of a person in a room 
can be obtained from the linear transformation" and "the linear transformation is 
used to obtain a sub-window of the image that is invariant to rotation and scale". 
The examiner finds no explanation with regard to maximizing the optimum 
trajectory of a person. 

b. Claims 2-14 and 18-19 are rejected as being dependent upon Claim 1. 

c. Claims 20 and 21 are rejected for the same reason given in Claim 1 . 
The following is a quotation of the second paragraph of 35 U.S.C 112: 

The specification shall conclude with one or more claims particularly pointing out and distinctly claiming the 
subject matter which the applicant regards as his invention. 

5. Claims 1-14 and 18-21 are rejected under 35 U.S.C. 112 2 nd paragraph. 

a. Claim 1 recites the limitation "the optimum trajectory" in line 8. There is 
insufficient antecedent basis for this limitation in the claim. 

b. Claims 2-14 and 18-19 are rejected as being dependent upon Claim 1. 

c. Claims 20 and 21 are rejected for reciting the same limitation. 

Claim Rejections - 35 USC § 103 
The following is a quotation of 35 U.S.C. 103(a) which forms the basis for all 
obviousness rejections set forth in this Office action: 
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(a) A patent may not be obtained though the invention is not identically disclosed or described as set forth in 
section 102 of this title, if the differences between the subject matter sought to be patented and the prior art are 
such that the subject matter as a whole would have been obvious at the time the invention was made to a person 
having ordinary skill in the art to which said subject matter pertains. Patentability shall not be negatived by the 
manner in which the invention was made. 

6. Claims 1-8 & 18-21 are rejected under 35 U.S.C. 103(a) as being unpatentable over Brill 

et al (U.S. Patent No. 6,542,621 Bl) in view of Brandstetter (U.S. Patent No. 5,185,815). 

a. Referring to Claim 1, Brill discloses processing a sequence of images (Brill at 

column 4, lines 59-60, discloses processing a succession of images provided by the video 

camera 12) to generate a statistical model (Brill at column 6, lines 5-8, teaches 

probabilistic templates or p-templates. The path of a p-template through the scene 

represents the track of a given person, which is maintained by the tracking system. Such 

probabilistic template is the claimed "statistical model") for each person to be tagged 

(Brill at column 6, lines 10-11, teaches the p-template is assigned to each of tracked 

objects in the video sequence. Note the objects herein are people as shown in figures 6A- 

6D), the statistical model incorporating at least one appearance feature (Brill at column 8, 

lines 22-23 discloses a p-template includes a raw color corrected probability. Note color 

is an appearance feature) and at least one geometric feature (Brill at column 7, lines 37- 

39, teaches the p-template also depends on the location of the tracked object. The location 

herein corresponds to the claimed "geometric feature") of the tagged person; applying the 

model to at least one subsequent image in order to perform at least one of a detection 

operation, a location operator and a tracking operation for the tagged person (Brill at 

column 8, line 34, teaches a p-template is applied for each tracked object, which implies 

the p-template is applied in a tracking operation); and controlling an action of the image 

processing system based on a result of the at least one operation (Brill at column 1, line 
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41, teaches applying such tracking technique in physical security environment, such as 
surveillance camera , which the tracking algorithm will direct the video camera to track 
the given objects). However, Brill does not teach the optimum trajectory of a person is 
maximized through linear transformation. Brandstetter at column 10, lines 18-36, teaches 
determining the object position in the correlation plane using linear transformation. At 
the time the invention was made, a person of ordinary skill in the art would have been 
motivated to do this because Brill at column 5, lines 1-10, teaches using the position in 
the image of the midpoint to identify the scale of a person would appear in the image, 
where Brill does not explicitly states the rotation of the image is invariant. Brandstetter at 
column 10, lines 18-36, teaches deriving the orientation of the tracking object in the 
correlation plane through use of linear transformation to achieve rotational invariance. 

b. Referring to Claim 2, Brill discloses wherein the sequence of image comprises a 
video segment (column 4, lines 59-60). 

c. Referring to Claim 3, Brill discloses wherein the processing step further includes 
processing the sequence of images to generate a plurality of statistical models, each of the 
models corresponding to a particular tagged person (Brill at column 6, lines 10-11 
teaches using plurality of p-templates when tracking objects in video sequence. Brill at 
column 8, line 34 teaches such p-template is generated for each tracked object). 

d. Referring to Claim 4, Brill discloses wherein the appearance feature comprises at 
least one of a color feature and a texture feature (Brill at column 8, lines 23-24, discloses 
color feature). 
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e. Referring to Claim 5, Brill discloses wherein the geometric feature comprises at 
least one of a region shape and a region position for a given one of a plurality of regions 
associated with the statistical model (Brill at column 7, lines 37-38, teaches the p- 
template depends on object locations, also see column 5, lines 64-67, and column 6, lines 
1-2). 

f Referring to Claim 6, Brill discloses wherein the statistical model is generated at 
least in part by segmenting a given image into a number N of different regions of similar 
appearance (Brill teaches at column 6, lines 10-11 teaches applying 2 p-templates, and 
they are segmented in different regions in a video image as shown in figure 5. And they 
have similar appearance, both are ovals). 

g. Referring to Claim 7, Brill discloses wherein the statistical model generated for a 
given person comprises a likelihood probability function which indicates the likelihood 
that the person is presented in a given image (Brill teaches generated p-template for a 
given object in a given video sequence, see column 6, lines 5-17) 

h. Referring to Claim 8, Brill discloses wherein the likelihood probability function 
P(I | Q) for person Q is computed as P(I \ Q) = ^P(R r \ Cl)P(r | Q) (Brill at column 

r=\,2,...N 

9, line 56, discloses use of the Bayesian probabilistic framework), where R r is a function 
of the at least one appearance feature and the at least one geometric feature (Brill teaches 
use of the appearance feature for p-template), and r is an index identifying one of N 
regions of similar appearance within the image I (Brill teaches calculating the p-template 
for a given object in a given video image). 
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i. Referring to Claim 18, Brill discloses wherein the controlling step comprises 
generating an output of the image processing system based on the result of the at least 
one operation (see explanation in claim 1). 

j. Referring to Claim 19, Brill discloses wherein the controlling step comprises 
altering an operating parameter of the image processing system based on the result of the 
at least one operation (Brill at column 9, lines 48-51, teaches when a new person enters 
the scene or a person leaves, the tracking algorithm will alter some parameter in the 
tracking operation). 

k. With regard to Claim 20, all the limitations are addressed in Claim 1. 
1. With regard to Claim 21, the only difference between Claim 21 and Claim 1 is 
Claim 21 calls for an article of manufacture comprising a storage medium for storing one 
or more programs. Brill at column 3, line 25 teaches using a computer workstation 13 to 
perform image processing, which inherently includes a storage medium for storing one or 
more programs. 

7. Claims 9-14 are rejected under 35 U.S.C. 103(a) as being unpatentable over Brill et al 
and Brandstetter as applied to claim 1 above, and further in view of Chang et al (U.S. Patent No. 
5,999,651). 

a. Referring to Claim 9, Brill discloses so-called statistical model generated for a 
given person comprises a likelihood probability function, which contains motion 
values as parameters. (Brill: column 3, lines 36-54 and column 7, lines 27-31) 
However, Brill does not explicitly distinguish the use of global motion and 
location motion. Chang at column 4, lines 17-37, teaches obtaining first contour 
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model using global motion estimation, and at column 5, lines 13-20, teaches 
performing location motion estimation based on the first contour model to obtain 
a second contour model. Therefore, the estimate probability described at column 
6, line 45, is a probability function based on global motion parameter and location 
motion parameter, which corresponds to the claimed "probability function" 
P(I |7\£,Q) . In addition, Chang at column 4, line 28, teaches 

d ( t ) = d ( t - 1 ) as ^ e c ' a i me d "linear transformation", v^t) at column 5, line 

14 is the claimed "a discrete variable". At the time the invention was made, it 
would have been obvious to a person of ordinary skill in the art to apply the 
probability estimation technique taught by Chang in Brill's system. One of 
ordinary skill in the art would have been motivated to do this because Chang and 
Brill both provide teachings for object tracking (even though Brill performs 
tracking on people, and Chang gives an example of tracking a bird on a lake, 
Chang does point out at column 1, line 19 that his system is also applicable to 
human), and using probability estimation to locate the object movements. 
Additionally, Chang provides local motion probability to detect the object 
movements more precisely, which reduces the error estimation rate, 
b. Referring to Claim 10, Chang discloses wherein a location of the person is 
determined using the linear transformation T (Chang at column 4, line 28 teaches 
d {t)- d {t ~ \ )^^ linear transformation). 
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c. Referring to Claim 11, Chang discloses wherein a pose of the person is 
determined using the discrete variable £ (Chang at column 5, line 14 teaches v^t) 

as the discrete variable). 

d. Referring to Claim 12, Chang discloses wherein the linear transformation T is 
used to obtain a sub-window of the image I that is invariant to rotation and scale 
(Chang discloses use of snake tracking, which is inherently invariant to rotation 
and scale). 

e. Referring to Claim 13, Chang discloses wherein the linear transformation T is 
implemented using a bilinear interpolation technique with a reference point x c , a 

rotation angle 6 , and a scaling factor s (Chang discloses reference point at 
column 4, line 25, the rotation angle and scaling factor are inherently included in 
snake tracking algorithm). 

f. Referring to Claim 14, Chang discloses wherein the local motion is modeled 
using a discrete set of states of the variable £ to capture M different poses of the 
person (Chang at column 5, line 14, discloses snake point v.(f)). 

Allowable Subject Matter 

8. Claims 15-17 are allowed. 

The following is an examiner's statement of reasons for allowance: 

a. Claim 1 5 defines a statistical model generated for a given person Q and image I 
comprises a likelihood probability function P(I |r,£,Q) = ^P(pix | 7\£,f2), 

pixel 

where r is an index to regions of similar appearance and N is a total number of 
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such regions, r=l,2,...N, and 

P(pix | r, 7, |, Q) = max | r, 7\ £ Q)/>(r | Q)] ,where P(pix | r, 7\ £ Q) is 
the probability of observing pixel pzx assuming that it belongs to an r-th region of 
the model on a pose £ , and P(r | Q) is the prior probability of the region at that 
pose. 

b. Claims 16-17 are dependent upon Claim 15. 

Conclusion 

9. Applicant's amendment necessitated the new ground(s) of rejection presented in this 
Office action. Accordingly, THIS ACTION IS MADE FINAL. See MPEP § 706.07(a). 
Applicant is reminded of the extension of time policy as set forth in 37 CFR 1 .136(a). 

A shortened statutory period for reply to this final action is set to expire THREE 
MONTHS from the mailing date of this action. In the event a first reply is filed within TWO 
MONTHS of the mailing date of this final action and the advisory action is not mailed until after 
the end of the THREE-MONTH shortened statutory period, then the shortened statutory period 
will expire on the date the advisory action is mailed, and any extension fee pursuant to 37 
CFR 1.136(a) will be calculated from the mailing date of the advisory action. In no event, 
however, will the statutory period for reply expire later than SIX MONTHS from the date of this 
final action. 

10. Any inquiry concerning this communication or earlier communications from the 
examiner should be directed to Tom Y Lu whose telephone number is (703) 306-4057. The 
examiner can normally be reached on 8:30AM-5PM. 
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If attempts to reach the examiner by telephone are unsuccessful, the examiner's 
supervisor, Leo H Boudreau can be reached on (703) 305-4706. The fax phone number for the 
organization where this application or proceeding is assigned is 703-872-9306. 

Information regarding the status of an application may be obtained from the Patent 
Application Information Retrieval (PAIR) system. Status information for published applications 
may be obtained from either Private PAIR or Public PAIR. Status information for unpublished 
applications is available through Private PAIR only. For more information about the PAIR 
system, see http://pair-direct.uspto.gov. Should you have questions on access to the Private PAIR 
system, contact the Electronic Business Center (EBC) at 866-217-9197 (toll-free). 



Tom Y. Lu 




TECHNOLOGY CENTER 2600 



